Augmented reality service negotiation

ABSTRACT

An augmented reality device engages in a mutual exchange of negotiated services with another device with a signal determined to have exhibited an area transition effect. The augmented reality device identifies, from a plurality of signals in the ambient environment, a subset of signals that have exhibited the area transition effect. From among the subset, the device identifies a signal that is indicative of support for the negotiation protocol. The negotiation is conducted with a second device associated with the corresponding signal.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/660,891, filed Apr. 20, 2018.

BACKGROUND

The advent of ubiquitous computing has resulted in an era in which largequantities of information about an individual are generated and stored.However, this information is rarely accessible to entities that mightmake use of the information. Moreover, when such information isavailable, the entity to which it pertains is often unable to controlthe entities to which the information is provided. Likewise, it is oftennot possible for devices controlled by various entities to interactbased on this information.

BRIEF DESCRIPTION OF THE DRAWINGS

Various techniques will be described with reference to the drawings, inwhich:

FIG. 1 illustrates an aspect of an environment in which an embodimentmay be practiced;

FIG. 2 illustrates an example of interchange of information betweenentities, in accordance with an embodiment;

FIG. 3 illustrates an example of a data sensitivity spectrum, inaccordance with an embodiment;

FIG. 4 is a flowchart illustrating an example of a process fornegotiating exchange of information and service participation, inaccordance with an embodiment;

FIG. 5 is a flowchart illustrating an example of a process forinitiating a negotiation, in accordance with an embodiment;

FIG. 6 illustrates an example of a mobile entity crossing a threshold,in accordance with an embodiment;

FIG. 7 is a flowchart illustrating an example of a process for obtainingand utilizing contextual information based on facial recognition, inaccordance with an embodiment;

FIG. 8 is a flowchart illustrating an example of a process forresponding to changes in an observed user environment, in accordancewith an embodiment;

FIG. 9 is a flowchart illustrating an example of a process foridentifying changes in an augmented reality scene;

FIG. 10 is a flowchart illustrating aspects of identifying changes in anaugmented reality scene, in accordance with an embodiment;

FIG. 11 is a flowchart illustrating an example process of servicenegotiation, in accordance with an embodiment;

FIG. 12 is a flowchart illustrating an example process of identifying asignal for service negotiation, in accordance with an embodiment; and

FIG. 13 illustrates an environment in which various embodiments can beimplemented.

DETAILED DESCRIPTION

Techniques and systems described below relate to negotiating potentialinteractions between an augmented reality device and one or more otherdevices, based on various factors. In one example, the identity of anindividual with whom information might be shared is determined via anidentification procedure, and a level of authorized interactiondetermined via a negotiation protocol. Interaction with the individual'saugmented reality device proceeds in accordance with the level ofauthorized interaction.

In an embodiment, an identity of a first entity associated with anaugmented reality device is determined with respect to the arrival ofthe first entity at a location associated with a second entity. In anembodiment, an arrival detection process first determines that the firstentity has arrived at the location. An initial level of a negotiationprotocol determines if the first entity consents to an identificationprocedure. If such consent is provided, the identity of the first entityis then determined. This process may be mutual, such that the secondentity may also indicate a willingness to be identified, and identifiedif willing.

As used herein, the term “mobile entity” may be used to refer to anindividual (or in some cases, a device such as a car) that is mobile.The term “location entity” is used to refer to an entity associated witha location. Generally, the location is fixed, but the term may be usedto refer to “places” such as cruise ships or airplanes.

In an embodiment, further levels of the negotiation protocol determine alevel of authorized participation between the devices or services of thefirst and second entities. Levels of the negotiation protocol exchange aminimal amount of information necessary to complete the negotiation. Thenegotiation protocol may include a series of information exchanges inwhich mutual consent for information exchange is determined. Thisconsent may be expressed, in an embodiment, as an agreed-upon set ofservices, such as programmatic interfaces, that each device makesavailable to the other.

In an embodiment, based on the authorized level of participation, eachof the first and second devices advertises services they provide orauthorize access to, and services they are willing to accept or consume.Interaction between the first and second entities then proceeds, basedon the advertised and accepted services. For example, in an embodiment,a first entity indicates a willingness to accept interactioncustomization, based on preferences or interaction patterns. The secondentity indicates that it provides interaction customization services. Inaccordance with embodiments of methods, systems, and computer programproducts described herein, a match between the accepted and offeredservices is detected, and the interaction of the first entity iscustomized via performance of the services.

An entity generally refers to an individual, group, organization,enterprise, or government. Depending on context, an entity may bemobile, e.g., a person or group of people arriving at a geographiclocation. In some embodiments, an entity such as a person may beassociated with another object, such as a car, bicycle, or airplane,which may be treated as linked to the person or group, or as atemporarily associated characteristic of the person or group. Likewise,and entity may be associated with one or more locations.

In embodiments, a service refers to a computer-implemented interface forperforming a computing function. The interface may be a standardized,defined, advertised, or discoverable means of interacting with acomputing device to perform the computing function(s) of the service.Examples include application programming interfaces, web services,wire-level protocols, and so forth. Augmented reality devices mayprovide services which may be invoked, given appropriate authorization,by another device or process. Services may also be hosted by cloud-basedproviders on behalf of an entity.

Embodiments of the present disclosure may be practiced in conjunctionwith Internet-of-things devices. The term Internet-of-Things (IoT)refers generally to devices which perform everyday tasks—such as coffeemakers, refrigerators, ovens, cars, and so forth—that send and receivedata via the internet. Other examples include fitness bands, watches,and so forth. These devices may broadcast information which may be usedto establish user identity, subject to the negotiation protocoldescribed herein. Once an agreed-upon level of information exchange isestablished, these devices may transmit information indicative of thepreferences of the user, such as menu items the user typically prefers,the room temperature preferred by the user, and so forth. The presentdisclosure describes how such information exchange may be practicedusing such devices. For example, when a user of a smartwatch arrives ata hotel room, the user's smartwatch might engage in the describednegotiation protocol with a thermostat in the hotel room. The thermostatmight then be provided with the user's desired temperature, andautomatically programmed with this preference.

A difficulty that may be encountered in communicating with such devices,and with other devices with whom service interchange might benegotiated, involves winnowing the signals of such devices from othersignals. At any given location, there may be a number of signalsavailable, many or most of which do not support the negotiable servicelevels described herein. As used herein, a signal may refer to wirelesssignals, or to other signal types. For example, in some instances asignal may be received as an observation made by a device with a sensor,such as a camera.

In an embodiment, devices supporting negotiable services are identifiedby identifying, from among a number of signals received by the receiver,those signals which are associated with an area transition effect. Asignal includes wireless communication, such as a WiFi or BLUETOOTH®signal. A transition effect, as explained herein, refers to a change toa change to a signal, as it is received by a device, when that device iscarried across a threshold. Examples of thresholds include doors,archways, and other building features. Having identified signalsexhibiting this effect, the threshold-crossing device then selects fromamong these signals a signal which includes information indicating thatthe source of the signal is a provider of a negotiable service. Anegotiation protocol may then be conducted, resulting in a determinationof what services each device can provide to the other.

In an embodiment, an arrival of a first entity at a location is detectedusing an adaptation of the BLUETOOTH® protocols. In an embodiment, thearrival of the first entity at the location is detected based on aprocess by which BLUETOOTH® beacon signals are selectively reduced to aviable set based on power signal analysis and location-based reduction.

A BLUETOOTH® beacon is a transmitter device that broadcasts a uniqueidentifier to other nearby devices. Typically, a BLUETOOTH® beacon is alow-energy transmitter. There may, nevertheless, be a large number ofbeacon signals in any given area. The described location processselectively reduces these signals to a set that for which it is viableto perform further analysis to determine whether the beacon isassociated with location or second entity, and to determine whetherservice-exchange negotiations should proceed.

In an embodiment, a mobile entity carries a device which comprises areceiver for signals broadcast from BLUETOOTH® beacons. From among thedetected beacons, a beacon associated with the location and a secondentity is identified, triggering a negotiation protocol.

In an embodiment, the obtained set of identifiers is reduced byaccessing a database of identifiers. For example, a database maycomprise a list of BLUETOOTH® identifiers and information indicatingidentifiers that are associated with negotiable services.

In an embodiment, the database is indexed by geographic location. Adevice held by the individual receives global positioning system (“GPS”)coordinates, or otherwise determines a location. The database is thenqueried to obtain a list of identifiers that are both located near thecurrent user and available for negotiation. The list is thencross-referenced with identifiers being broadcast by nearby beacons,thereby obtaining a candidate list of negotiation partners.

Typically, identifiers with higher power represent nearer beacons.However, there may be an area transition effect in which the signalsfrom beacons in a nearby area are received with less power prior to orduring an area transition, and with relatively more power subsequent tothe transition. For example, an identifier may be received with lowerpower as the user & receiver enter a doorway, and then received withhigher power after the doorway has been crossed.

In an embodiment, filtering is applied to the received beacon signals toestimate a probability that an associated identifier is representativeof a nearby beacon. For example, Kalman filtering may be used tocompensate for or supplements the aforementioned transition effect. Avariety of filtering techniques may be employed to reduce the set ofsignals that are analyzed. Filtering may be applied in stages, e.g.prior to analyzing signals for a transition effect, and again oncesignals exhibiting such effects have been identified.

In an embodiment, direction of movement observations are used toincrease accuracy of the determined set of identifiers. For example,direction of movement observations may be obtained via GPS readings,inertial sensors, and so on. Based on direction of movement observationsand observations of area transition effects (as described above), it isdetermined whether an identifier is associated with a space that isbeing left or entered.

In an embodiment, movement observations are used to determine whetheridentifiers are associated with a space that is likely to be entered.For example, it may be determined that the individual typically enters aparticular space following a preceding pattern of movement. For example,it might be observed that an individual typically enters a particularestablishment after first moving into another location, such as ahallway or parking lot.

In an embodiment, the arrival of an entity at a location is detectedbased on an identifier broadcast by a device carried, worn, or otherwiseborne by an individual. The identifiers are read by a receiving deviceat the location.

In an embodiment, the arrival of an entity at a location is detectedusing recognition techniques, such as facial recognition technology. Insuch embodiments, facial recognition is employed to identify anindividual as the individual arrives at a location or is presented to acamera.

In an embodiment, the arrival of an entity at a location is detectedusing audio or biometric technology, such as voice printing,fingerprinting, retinal scanning, and so forth. These techniques are, invarious embodiments, combined to improve detection accuracy over amethod used individually.

In an embodiment, a negotiation protocol is carried out on behalf of afirst entity that has arrived at a location and a second entity that isassociated with the location. For example, the first entity maycorrespond to an individual and the second entity may correspond to anorganization.

In an embodiment, the negotiation protocol comprises a series ofinterchanges, each of which involves a minimal transfer of sensitiveinformation. For example, in an embodiment, a series of informationexchanges occurs. During each exchange, only information that has beenpreviously authorized for disclosure is exchanged. The authorization mayoccur during an earlier interchange. For example, a first level exchangemight determine whether a given individual has consented to or ispermitted to participate in further negotiation. If so, a second levelof exchange might occur, in which more information, such as informationthat might be necessary to validate the individual's and enterprise'sidentities, might be exchanged. Then, a third level of exchange mightoccur, such as identifying and invoking available services.

In an embodiment, a zero-knowledge protocol, sometimes referred to as azero-knowledge proof, is employed. A zero-knowledge protocol is a methodof exchanging information in which knowledge K is provided by a firstparty (the “prover”) is verified by a second party (the “verifier”), andin which no information other than the fact that the provider is inpossession of knowledge K. In an embodiment, each entity is a prover ina first application of a zero-knowledge protocol and a verifier in asecond application of a zero-knowledge protocol.

In an embodiment, a series of zero-knowledge proofs are undertaken, inwhich increasing levels of information are verified in each exchange. Ineach exchange, each party provides information which does not exceed athreshold sensitivity, but which is sufficient to determine whether thethreshold should be increased in the next exchange.

In an embodiment, following application of one or more zero-knowledgeproofs, the negotiation process continues by exchanging informationabout available services, and invoking one or more of the availableservices. This process may repeat, depending on the outcome of eachservice invocation, to permit further services to be exposed.

For example, in an embodiment, the first and second entities eachadvertise a credential service through which validation credentials maybe exchange. The service is provided after an initial zero-knowledgeproof of each entity's willingness and ability to participate in anegotiation process. After the zero-knowledge proof, each entity mayinvoke the other's respective service to obtain credentials, andvalidate the credentials through a third-party validation service. Uponcompletion of this process, a further set of services is advertised andinvoked. For example, a device of the first entity (e.g., theindividual's mobile phone) might advertise the individual's preferences,and a device of the second entity (e.g., a computing device at thelocation) might advertise a service which can customize the individual'sexperience at the location, a service which can provide informationabout the location, and so on.

In an embodiment, various information and computing services providersstore data and operate services on behalf of the entities. Theseservices may be described herein using the term entity services, entitydatabases, and so forth. Access to these services may be controlled viathe negotiation protocol and the services advertised by an entity'sdevice.

Entity services may provide a variety of information and interface. Inan embodiment, and entity service maintains a store of preferenceinformation. The preference information might include favorite foods,preferred room temperature, subjects of interest, and so forth. In othercases and embodiments, entity services might comprise travel history,locations recently visited, and so on. Entity services may also maintainvarious biographical information, such as name, address, age, education,and so forth. However, entity services do not disclose informationwithout at least performing the negotiation protocol described herein.

In an embodiment, an entity service stores information comprising“cookies” stored at the request of an entity. The information may bepersistent, may expire after a predetermined period, or may be deletedby the entity to which the information pertains.

In an embodiment, a location entity stores information pertinent to amobile entity visiting the location. The location entity can use theinformation, for example, to retain preferences of the mobile entitybetween visits to the location, to store a history of transactionsconducted at the location, and so forth. The location entity may storethis “cookie” of information after negotiating with the mobile entity'sdevice to determine if such storing is authorized by the mobile entity.

In an embodiment, information associated with the mobile entity isshared between multiple location entities. Access to the information ismade subject to constraints imposed by the mobile entity's securitypreferences, as obtained from the negotiation protocol.

In an embodiment, preferences provided by a mobile entity to a firstlocation entity are made available to a second location entity. Forexample, the mobile entity might express a preference for food choices,such as how much spice the entity prefers, which can be stored by thefirst location entity and accessed by a second location entity in orderto customize the mobile entity's experience at the second location. Thesecond location entity might, for example, offer food choices having asimilar spiciness level as were indicated by the first location entity.

In an embodiment, data provided by entities is synchronized. In anembodiment, there are, for a given mobile entity, a plurality of datacollections. Each of the plurality of data collections may pertain to alocation entity. For example, a first data collection might pertain to afirst location, and a second data collection might pertain to a secondlocation. In an embodiment, synchronization involves copying databetween the collections, while resolving conflicts as necessary. As anexample, a user preference in one collection can be mapped to a userpreference in a second collection.

In an embodiment, a normalization process is applied to data receivedfrom or provided to an entity. Normalization relates to adjusting datato facilitate meaningful interchange of information. For example, someentities might rate a quality on a scale of 1 to 10, while others mightrate a quality on a scale of 1 to 5. A rating value of 5 means verydifferent things in the two scales. Accordingly, a normalization processmight adjust or translate these ratings to a standardized scale, inorder to facilitate interchange of the rating information betweenentities. The normalization may further be conducted based on data fromother entities, such as other mobile entities. For example, a firstmobile entity might rate food as “2” on a five-star spiciness scale, ata location where other entities tend to rate the food as a “4.” Thefirst entity's perception of the scale may therefore tend to be somewhatlow. Suggestions and preferences may then be based on this knowledge,perhaps by offering the entity spicier food. In an embodiment, aclassifier, such as an artificial neural network, Bayesian classifier,support vector machine, and so forth, is trained to perform thenormalization.

Embodiments of the present disclosure may be applied in a variety ofcircumstances to perform a variety of functions, based on the disclosedtechniques for negotiating service availability and serviceparticipation.

In an embodiment, a negotiation process identifies facts an entity iswilling to share, and conditions for sharing facts. For example,information about an individual may be restricted to informationpertinent to the entity's current location. Information such as theentity's location may be considered to be part of the context of atransaction. As used herein, a transaction refers to a potential,predicted, ongoing, or concluded exchange or interaction betweenentities. For example, a transaction might be used to refer to apotential exchange between a mobile entity and a location entity thatbegins when the first entity arrives at the location. More generally,transactions may be related to sessions of information sharing andservice exchange.

In embodiments, a result of the negotiation process is a set of serviceswhich are made available by the respective devices. The negotiationprocess determines which services, of those implemented by the device,are to be made available to the other device. The set of services thatare made available includes those that are consistent with a permissiblelevel of information sharing, and excludes those which are not.

In an embodiment, a service makes facts about an individual available toa location entity in order to facilitate or enhance a transaction. Thefacts may include, in various combinations, information about foodallergies, menu preferences, temperature preferences, lightingpreferences, music preferences, and so forth.

In an embodiment, a service makes facts about an individual'stransaction history available to a location entity. The facts mayinclude, in various combinations, prior transactions at the samelocation, prior transactions at other locations, a history ofproduct-specific questions asked by the entity, and so forth.

In an embodiment, preference information is aggregated to arrive at agroup consensus preference. For example, each of a number of individualsat a location (i.e., a number of mobile entity devices) providespreference information regarding a music selection. Based on aggregatesof the music preferences, a conforming music playlist is selected. Inanother embodiment, environmental factors such as lighting andtemperature are selected. This consensus preference information may alsobe made available as a service.

In an embodiment, services are provided to facilitate a group search orgroup consensus-forming. For example, each of a plurality of mobileentity devices may provide preference information related to foodpreferences. Based on an intersection of the preferences, a list ofsuggested nearby restaurants is provided.

In another embodiment, a service exchanges information pertinent toavailability for employment. Related services may also be madeavailable, including a credential verification service, a calendarservice to indicate times the entity is available for employment, and aprice negotiation service which may, for example, automate acceptance ofa job at a negotiated price.

In an embodiment, services are offered to provide incentives. In afurther aspect, incentives are offered to a plurality of entities inorder to steer a desired outcome. Examples of a desired outcome includeencouraging movement of individuals to another location (i.e., from thehardware department to the clothing department, or from a crowded bar toa nearby restaurant that has seating available).

In an embodiment, advertisements and promotional information are offeredto mobile entities, in response to authorization of such services andthe exposure, by the mobile entity's device, of services which enablethe provision of these services.

In an embodiment, payments are completed via services offered by thelocation entity and/or mobile entity.

In an embodiment, an auction is conducted via an auction participationservice of a mobile entity, interfacing with an auction conductionservice of a location entity.

In an embodiment, an elevator is summoned responsive to the arrival ofan individual at a location. For example, a location entity might offera service to hail elevators. This service might be made availablefollowing a negotiation protocol in which the identity of a mobileentity is determined. The service might further automate or facilitatethe selection of a destination floor for the elevator, based on securityinformation provided by a service made available by the mobile entity'sdevice.

In an embodiment, a device may provide one or more services which informthe other device concerning available services. These queries may besubject to constraints imposed by the negotiation process. For example,services which exceed a mutually agreed-upon sensitivity level may behidden in the results of such queries.

In an embodiment, participation in or offering of services may becontext-dependent or conditional. For example, a preference serviceoperating on behalf of a mobile entity may only be offered while thecorresponding entity is at a selected location. When the entity leavesthe location, the offering is terminated. Likewise, a location entitymay permit participation in a service only to mobile entities currentlyat the location. In an embodiment, the ability to use a service expiresafter a period of time subsequent to leaving a location.

In the preceding and following description, various techniques aredescribed. For purposes of explanation, specific configurations anddetails are set forth in order to provide a thorough understanding ofpossible ways of implementing the techniques. However, it will also beapparent that the techniques described below may be practiced indifferent configurations without the specific details. Furthermore,well-known features may be omitted or simplified to avoid obscuring thetechniques being described.

Techniques described and suggested in the present disclosure improve thefield of computing, especially the fields of device-to-deviceinteraction and security, by proposing location-sensitive, escalatinginformation exchange. Additionally, techniques described and suggestedin the present disclosure improve the function of network-connecteddevices by proposing techniques for more secure discovery of servicesprovided by networked devices. Moreover, techniques described andsuggested in the present disclosure are necessarily rooted in computertechnology in order to overcome problems specifically arising in thefield of networked devices.

FIG. 1 illustrates an aspect of an environment 100 in which anembodiment may be practiced. Specifically, FIG. 1 depicts anetwork-accessible assemblage of services, by which interaction betweena device 108 of a mobile entity 106 and a device 104 of an entityassociated with a location 102 occurs. Note that although FIG. 1 depictsa single mobile entity's device 108 and a single location entity'sdevice 104, in various embodiments a plurality of such devices may bepresent and may participate in a service negotiation process. The mobileentity's device 108, in various embodiments, may be any of a variety ofaugmented reality devices, such as augmented reality glasses,smartphones, projectors, goggles, and so forth.

The location 102 may be associated with an entity, referred to herein asa location entity. For example, the location 102 may be a store,airport, shopping mall, school, government building, and so forth. Thelocation entity is an entity associated with the location 102.

A device 104 may be placed in the location 102 by the location entity.The location entity's device 104 transmits a location identifier whichis received by the mobile entity's device 108. The mobile entity'sdevice may be a computing device with a receiver suitable for detectingthe transmitted identifier, or a receiving device which may be connectedat the location 102 to a computing device.

In an embodiment, the mobile entity's device 108 transmits an identifierread by the location entity's device 104. In other cases andembodiments, both the location entity device 104 and the mobile entitydevice 108 transmit an identifier.

In an embodiment, the services 112-120 are accessible to the devices104, 108 via a network 110. The network 110 may, for example, comprise awireless network and the Internet. The services 112-120 may be invokedusing various protocols, such as various web service, simple objectaccess protocol (“SOAP”), or representational state (“REST”) protocols.These examples are not intended to be limiting, and as such should notbe construed in a manner which would limit the scope of the presentdisclosure to only those embodiments which include the specific examplesprovided.

In an embodiment, the identification service 112 determines identityinformation based on data sent from the mobile entity's device 108and/or location entity's device 104. The identity information comprisesinformation used to enable negotiation to proceed, and may exclude otherinformation, and in particular other information which may be consideredsensitive or subject to regulatory restrictions.

In an embodiment, the negotiation service 114 facilitates a negotiationbetween the location entity device 104 and the mobile entity's device108. The negotiation, in an embodiment, is performed by hosted servicesexternal to the devices 104, 108. As described herein, the negotiationcomprises a series of interchanges in which increasing amounts ofinformation is exchanged. An example is provided by FIG. 4.

The services 116 refers to a variety of hosted computing services thatmay operate on behalf of the mobile entity 106, the entity associatedwith the location 102, or other entities, such as a third party. Theservices 116 may operate on a device, such as an augmented realitydevice, or on a hosted computer system. Examples of services include aweb service interface for providing preferences of the mobile entity106, a web service interface enabling a device or devices of thelocation entity to receive the mobile entity's 106 preferences,interfaces permitting environment of the location 102 to be customizedbased on the preferences, and so on.

The entity information services 118 are another example of a service. Inan embodiment, information about entities is stored and normalized tofacilitate interchange of persistent information about an entity. Thesefunctions are performed by the entity information services 118.

The validation services 120 are an example of a service which may beaccessed during negotiation or at other times. In an embodiment, thevalidation services 120 are a trusted third-party capable of providingidentity verification services. In another example, the validationservices 120 are a trusted third-party capable of validating otherinformation, such as qualifications to perform a task.

In an embodiment, a multi-entity service negotiation is performed. Thismay proceed similarly to a negotiation between the entity's devices 104,108 as depicted in FIG. 1, except that mutual consent to informationexchange is made between each of the respective devices.

FIG. 2 illustrates an example 200 of interchange of information betweenentities. In the example 200, a mobile entity 206 interacts, consistentwith the techniques proposed herein, with a first location entity 202. Aresult of the interaction is data recorded in a personal cookie 204 a.Here, “personal cookie” refers to data pertaining to the mobile entitythat is persistent and which may, subject to a negotiation process, beshared with another entity such as a second location entity 212.

Data in the personal cookie 204 a is subjected to a normalizationprocess 214 and stored in a database 208. The normalization 214 adjustsdata in the personal cookie 204 a to facilitate interchange between theentities 202, 212, by converting data in the personal cookie 204 to aformat and/or scale that can be understood by other entities.

The database 208 records data from the personal cookie 204 a. The datamay be provided, as personal cookie 204 b, to the second location entity212 if authorized by a negotiation process 210.

FIG. 3 illustrates an example 300 of a data sensitivity spectrum 308. Inan embodiment, access to an entity's data is restricted in accordancewith the data sensitivity spectrum 308. As illustrated by FIG. 3, datamay be categorized according to levels of sensitivity. Some data may beconsidered to be always sharable. For example, an identity beacontransmitted by a mobile entity's device may be considered alwaysshareable. Similarly, data which may be visually captured in order tofacilitate the first level of negotiation may also be considered alwayssharable. In some instances, some preference information, such as afavorite color, might be considered mostly shareable unless anindividual has opted out of the system entirely. Other information maybe considered sensitive, or highly sensitive, and is shared only underspecific, highly controlled circumstances.

In an embodiment, other data may be sharable depending on context, suchas the identity of the other party, the location of the entities, or thetransaction being conducted. For example, as described herein, aninformation sharing session may comprise a period of time in whichcertain forms of information—related to the purpose of the session—maybe shared. Various contextual information, such as that which may begleaned by identifying cues in the environment of an augmented realitydevice, may supply such context. For example, a purpose of a visit to astore might be inferred from analysis of the images of objects capturedby the camera of an augmented reality device. Other examples ofcontextual information include location data from a global positioningsystem, calendar data from the entity, and so forth. Note that a fullanalysis of context may require access to information from differentlevels of the sensitivity spectrum. Embodiments may therefore performseveral levels of contextual analysis during a negotiation of services,in stages, when and if access to progressively more sensitive data isauthorized.

Certain levels of information in the spectrum 308 may be shared based onvarious levels of verification or authorization. For example, someinformation may be shared based on an initial, zero-knowledge proof 302.Other information might be exchanged after an account or othercredentials are verified through credential exchange 304. For moresensitive information, a trusted third-party 306 can be used. Forexample, the trusted third-party 306 might be used to verify thecredentials of each party.

FIG. 4 is a flowchart illustrating an example of a process 400 fornegotiating an exchange of information and service participation, inaccordance with an embodiment. Some or all of the process 400 (or anyother processes described, or variations and/or combinations of thoseprocesses) may be performed under the control of one or more computersystems configured with executable instructions and/or other data, andmay be implemented as executable instructions executing collectively onone or more processors. The executable instructions and/or other datamay be stored on a non-transitory computer-readable storage medium(e.g., a computer program persistently stored on magnetic, optical, orflash media).

For example, some or all of process 400 may be performed by any suitablesystem, such as a server in a data center, by various components of theenvironments described herein, including those depicted in FIG. 1, andin FIG. 13, such as the one or more web servers 1306 or the one or moreapplication servers 1308, by multiple computing devices in a distributedsystem of a computing resource service provider, or by any electronicclient device such as the electronic client device 1302. The process 400includes a series of operations which, except where logically required(for example because the input to one step requires the output of aprevious step, may be performed in an alternative order, or performed inparallel. In some embodiments, certain steps may be omitted.

In 402, the arrival of a mobile entity at a location is detected, wherethe location is associated with a second entity. For the purpose of thisexample, the second entity may be referred to as a location entity, butin other cases and embodiments the second entity may be another mobileentity, or some other entity.

In 404, negotiation is initiated. Negotiation refers to a series ofinteractions in which it is determined what types of information may beexchanged between the entities and what types of services are availableand authorized.

In 406, a decision is made regarding whether negotiation may commence.Based on various factors, negotiation may be prohibited. For example,based on facial recognition technology, it may be determined that themobile entity is an individual that is too young, based on laws andregulations, to collect information from. In another example, the entitymay have previously expressed a desire to be excluded from the process.Rather than performing facial recognition, other techniques may be used,such as the identification of a non-consent code in a signal sent fromthe mobile entity's device. Alternatively, the mobile entity begins theprocess with an initial request to negotiate service exchange.

If the negotiation is permitted, in 408 zero-knowledge proofs areperformed to confirm basic information about the identity of theparties. For example, both entities' devices may submit, to each other,a zero-knowledge proof of their ability to participate in a servicenegotiation process, which is then verified by the other party. This maythen escalate to an exchange and verification of credentials.

In 410, it is determined whether the identities have been sufficientlyproven. If so, a process of service publishing and service consumptioncontinues in 412 and 414. In some cases, a first level of serviceinteraction must occur before a subsequent level. For example, an agentof the mobile entity might publish a conditional preferences services,which might then be consumed by an agent of the location entity. If theconditions are met, an agent of the mobile entity might then publish aset of services which make available various types of preferenceinformation. The services published comprise those which the respectivedevices or entities are authorized to use.

In 416, it is determined whether there is continued authorization toparticipate in service publishing and consumption. Authorization mightbe terminated based on various factors, such as the results of priorservice invocations, a change in location, and so on. In variousembodiments, the end of an information sharing session is determinedfrom a change in context, such as a departure of one of the involvedentities, the passage of time, the end of a scheduled time period, theconclusion of a transaction, and so forth.

FIG. 5 is a flowchart illustrating an example of a process 500 forinitiating a negotiation, in accordance with an embodiment. Some or allof the process (or any other processes described, or variations and/orcombinations of those processes) may be performed under the control ofone or more computer systems configured with executable instructionsand/or other data, and may be implemented as executable instructionsexecuting collectively on one or more processors. The executableinstructions and/or other data may be stored on a non-transitorycomputer-readable storage medium (e.g., a computer program persistentlystored on magnetic, optical, or flash media).

For example, some or all of process may be performed by any suitablesystem, such as a server in a data center, by various components of theenvironments described herein, including those depicted in FIG. 1, andin FIG. 13, such as the one or more web servers 1306 or the one or moreapplication servers 1308, by multiple computing devices in a distributedsystem of a computing resource service provider, or by any electronicclient device such as the electronic client device 1302. The processincludes a series of operations which, except where logically required(for example because the input to one step requires the output of aprevious step, may be performed in an alternative order, or performed inparallel. In some embodiments, certain steps may be omitted.

In 502, an identification service, such as the identification servicedepicted in FIG. 1, receives identifiers of a mobile entity and alocation entity. The identification service may, in embodiments, possessa superset of the information possessed by each respective party, butdoes not permit a service operating on behalf of one party access todata associated with the other, until negotiation has completed.

In 504, the identification service locates agents that may negotiate onbehalf of the mobile entity and the location entity. In an embodiment,the agents are implemented as hosted services accessible via a network,such as the internet. In another embodiment, an agent is implemented ona device of the mobile entity and/or the location entity.

In an embodiment, the agents are located by using a hosted databaseservice to map from the identifiers to agents of the respective parties.

In 506, the negotiation is initiated between the located agents. Asdescribed herein, the negotiation comprise a series of exchanges, eachof which involves progressively more sensitive data and each if which isperformed only upon the mutual consent of both parties. For example, inan embodiment, the negotiation comprises mutual zero-knowledge proofs,mutual credential exchange and validation, and exchange of availableservices.

At 508, depending on the outcome of the negotiation, the servicesprovided by each entity's device (or provided by the agents on behalf ofthe entities) are exchanged between the two agents. For example, amobile entity's agent might advertise services that provide preferenceinformation, while a location entity's agent might advertise servicesthat enable the completion of a transaction. Note that some of theseservices might be invoked by directly contacting the entity's device,while others might be invoked by contacting a hosted service. Theadvertisements of the services, therefore, include information such asinternet-protocol addresses, formats, and other information necessary toinvoke the service. Further, a cryptographic key or other data might beprovided to allow the other entity to invoke the service, whilepreventing third party interference.

A service negotiation may be made in response to the identification of adevice with which the service negotiation may be conducted. It may bedifficult, however, to determine when to look for such devices, or toefficiently locate a signal from such a device, given the preponderanceof other signals in the wireless spectrum.

FIG. 6 illustrates an example 600 of a mobile entity 604 crossing athreshold 608. The mobile entity 604 may, for example, be entering aroom from outside, or from another room. For the purposes of the example600, it may be presumed that there are two beacons 602, 606. The firstbeacon 602, for the purpose of the example 600, is on one side of thethreshold 608, while the other beacon 606 is on the opposite side. Asdepicted in FIG. 6, the mobile entity 604 is crossing the threshold 608.Consequently, in the example 600 the beacon 606 is considered to be ofinterest to mobile entity 604, whereas the beacon 602 may be of lessorinterest, or of no interest, to the mobile entity 604. Described hereinare techniques for differentiating between signals broadcast by the twobeacons and determining which signals should be processed, for thepurpose of initiating service negotiation.

For example, using the differentiation techniques described herein, anidentifier or other information broadcast by the beacon 606 may beobtained efficiently, even in view of a preponderance of extraneoussignals. Once the beacon has been identified, the described techniquesfor negotiation and service interchange may then be performed.

In cases and embodiments, service negotiation and information sharingmay be performed in conjunction with various facial-recognitiontechniques, and in particular with techniques oriented to a databasefrom which the identity of an entity can be determined, based onvisually identifiable facial characteristics. This database may becontext-dependent, so that its data is accessible to a party only inspecific contexts, such as during a meeting with colleagues or otherknown or consenting individuals.

For example, an augmented reality device may display information relatedto faces identified within an environment, such as within a meetingroom. In an embodiment, an augmented reality device captures image datacontaining faces, and transmits the data to a cloud-based serviceprovider. The cloud-based service provider identifies facts associatedwith the captured faces, and transmits the facts back to the augmentedreality device for display. Using a position and orientation adjustmenttechnique, relevant facts are displayed near the associated face. Thisinformation may include, for example, the person's name, facts about theindividual, possible conversation prompts, and information about priorinteractions with the person.

In an embodiment, the described techniques for negotiation and serviceinterchange are performed with the identified individual, and thenservices, facts, or preferences authorized by the individual for sharingare displayed. Moreover, the described techniques for servicenegotiation may be performed using context derived from theidentification of the individual.

In an embodiment, groups of faces are analyzed. Facts or analysisresults, such as sentiment analysis results, are displayed. For example,in an embodiment group affiliations are displayed. In anotherembodiment, properties common to members within view are displayed.Various visual indications may be displayed to indicate facts pertinentto groups rather than individuals, such as shading, outlines, and soforth. This information may also be used to supply context for theservice negotiation process. For example, in embodiments, the set ofservices made available to the respective parties in a negotiationprocess may depend on the identity or recognition of the faces.

FIG. 7 is a flowchart illustrating an example of a process for obtainingand utilizing contextual information based on facial recognition, inaccordance with an embodiment. Some or all of the process 700 (or anyother processes described, or variations and/or combinations of thoseprocesses) may be performed under the control of one or more computersystems configured with executable instructions and/or other data, andmay be implemented as executable instructions executing collectively onone or more processors. The executable instructions and/or other datamay be stored on a non-transitory computer-readable storage medium(e.g., a computer program persistently stored on magnetic, optical, orflash media).

For example, in embodiments, some or all of the process may be performedby an augmented reality device. In embodiments, some or all of processmay be performed by any suitable system, such as a server in a datacenter, by various components of the environments described herein,including FIG. 13, such as the one or more web servers 1306 or the oneor more application servers 1308, by multiple computing devices in adistributed system of a computing resource service provider, or by anyelectronic client device such as the electronic client device 1302. Insome instances, actions are performed by a hosted process acting as anagent of the augmented reality device. As such, some actions attributedto the device may instead be performed, in whole or in part, by a hostedservice.

The process 700 includes a series of operations which, except wherelogically required (for example because the input to one step requiresthe output of a previous step, may be performed in an alternative order,or performed in parallel. In some embodiments, certain steps may beomitted.

In 702, a database of faces is maintained. The database may, forexample, be maintained by a hosted service, or by computing devices orservers operating in a data center. In embodiments, an enterprisemaintains a database comprising data for individuals known to it, suchas the employees of the enterprise. In an embodiment, the databasecomprises facial image data. In another embodiment, the databasecomprises representational data, as used by associated facialrecognition techniques.

In 704, a registry or database of facts is maintained, indexed bybiometric signatures such as facial identity characteristics, and/or byidentity information corresponding to the biometric signature. Forexample, in an embodiment, the database is indexed by facialcharacteristic data, permitting efficient lookup of facts associatedwith a given face. In another embodiment, a database is indexed by theidentities of individuals represented in the database of faces.

In 706, an image comprising a face is received. The image may, forexample, be obtained through a camera or cameras associated with anaugmented reality display. In an embodiment, the augmented realitydisplay is a smartphone. In another embodiment, the display is comprisedin augmented reality glasses, or in an augmented reality headset, or ina projector. In some embodiments, the processing is done remotely to thedevice which contains the display. For example, the process 700 may beperformed by a hosted computing service accessible via the Internet. 706thus may depict receiving image data from an augmented reality device,via the Internet. In other embodiments, the augmented reality devicecontains the recognition data, and performs the facial recognitionprocessing locally.

In 708, the characteristics of the face, or an identity determined fromthe face, is mapped to various facts. Note that the depicted person maynot be completely identified. Instead, it may be sufficient, in certaincases and embodiments, to determine if this person is recognized andconsistent with other contextual information. For example, instead ofdetermining an individual's identity, it might be sufficient todetermine that the person is a recognized employee of a business anentity is dealing with. The recognition of an individual (meaning thatthe individual is known to have been involved in a previous transaction,as opposed to knowing the identity of the individual) may supplycontextual information for a service negotiation. Likewise, factors suchas the determination of the identity of the individual, confirmation ofthe role of the individual in a transaction, and so forth, may supplycontextual information for conducting the service negotiation.

At 710, information about the person is used to enhance the augmentedreality experience. This may comprise performing a service negotiation,generate and displaying information on an augmented reality device, andso on.

One challenge in providing augmented reality experiences involveshandling latency, particularly in view of the user's changingenvironment. One example involves the proper alignment of informationdisplayed in an augmented reality scene. For example, an augmentedreality device might display information about various individualswithin the scene. Further, it might attempt to display that informationnear the relevant individual, so that the user knows who the dataapplies to. This may, however, be difficult to do because of themovement of the user and the observed individuals.

Such movement may also be relevant to other scenarios, such asestablishing context for the negotiation of services. For example, anindividual might be observed passing through the user's field of vision.If so, this individual should, in most cases, not be used to establishthe context of the negotiation.

FIG. 8 is a flowchart illustrating an example of a process 800 forresponding to changes in an observed user environment, in accordancewith an embodiment. Some or all of the process (or any other processesdescribed, or variations and/or combinations of those processes) may beperformed under the control of one or more computer systems configuredwith executable instructions and/or other data, and may be implementedas executable instructions executing collectively on one or moreprocessors. The executable instructions and/or other data may be storedon a non-transitory computer-readable storage medium (e.g., a computerprogram persistently stored on magnetic, optical, or flash media). Forexample, some or all of process may be performed by any suitableaugmented reality device, such as a wearable computing device, i.e.,augmented reality glasses. The process includes a series of operationswhich, except where logically required (for example because the input toone step requires the output of a previous step, may be performed in analternative order, or performed in parallel. In some embodiments,certain steps may be omitted.

In 802, an image or other representative data is transmitted from theaugmented reality device to a remote service for processing. Thetransmission and processing may take a relatively long amount of time,and as such can result in significant latency between when the image istransmitted and when data based on the image is returned to the device.The image may comprise the augmented reality scene as observed by thedevice, or may comprise portions of such images, such as a depiction ofa face. In embodiments, rather than a complete image, various compressedformats are used. In other embodiments, data representing particularcharacteristics, such as facial identity characteristics, is sent.

In 804, the augmented reality device maintains a buffer of position andorientation information. In an embodiment, the information is maintainedover a time sufficient to account for the latency of processing theimage data. As the device moves, and as people and things in theaugmented reality scene move, the device stores information to describethe movements that have taken place.

In 806, after a period of latency, facts pertinent to the transmittedimages are received. During this time, the position and/or orientationof the augmented reality device may have changed. As such, in 808, acurrent on-screen position of the face is estimated using changes inposition and/or orientation since the image was transmitted. Thisprocess relies on the buffer, which contains a history of movements. Thedevice may use information in the buffer to trace the path of theidentified object within the scene, and calculate an estimated positionof the identified face or object.

In 810, the received facts are used to enhance the augmented realityexperience. In an embodiment, the facts are displayed in the augmentedreality scene, as viewed by the device's user, near the estimatedposition of the face. The display can then be updated periodically asthe view orientation or position of the face changes. In anotherembodiment, the facts are used to establish context for a servicenegotiation. For example, the system may determine whether the factsapply to an individual or object that is still in the scene, and if so,use that information to establish context. If not, the information canbe disregarded, or in some cases used to determine if information thathad been used to establish context should no longer be used.

Contextual cues for a service negotiation may sometimes be derived fromchanges in the scene viewed by an augmented reality device. FIG. 9 is aflowchart illustrating an example of a process 900 for identifyingchanges in an augmented reality scene, in accordance with an embodiment.Some or all of the process (or any other processes described, orvariations and/or combinations of those processes) may be performedunder the control of one or more computer systems configured withexecutable instructions and/or other data, and may be implemented asexecutable instructions executing collectively on one or moreprocessors. The executable instructions and/or other data may be storedon a non-transitory computer-readable storage medium (e.g., a computerprogram persistently stored on magnetic, optical, or flash media).

For example, some or all of process may be performed by any suitablesystem, such as a server in a data center, by various components of theenvironments described herein, including FIG. 13, such as the one ormore web servers 1306 or the one or more application servers 1308, bymultiple computing devices in a distributed system of a computingresource service provider, or by any electronic client device such asthe electronic client device 1302. The process includes a series ofoperations which, except where logically required (for example becausethe input to one step requires the output of a previous step, may beperformed in an alternative order, or performed in parallel. In someembodiments, certain steps may be omitted.

In 902, a representation of an augmented reality scene is repeatedlycaptured. The representation may be in any format suitable for thefurther processing steps described.

In 904, changes between scenes are identified. This may comprise anencoding step in which raw image data is converted to a differentialrepresentation. A filtering process may identify frames in the sequencein which there are large differentials. These may be indicative ofnotable scene changes.

In 906, the scene changes are classified. Classification comprisesidentifying changes which may be understood to be important, such aswhen a person enters or exits a scene. This step may therefore involveidentifying changes in the content, rather than appearance, of thescene. For example, some scene changes may involve changes to lightingconditions or other factors, but not involve any changes to content.Similarly, significant scene changes may occur when the user of anaugmented reality device crosses a threshold, e.g., by entering abuilding.

In 908, information is provided based on the classification. Theinformation can comprise useful data, such as a time at which a personarrived at a location, and so forth, a time at which a person left alocation, and so forth.

FIG. 10 is a flowchart illustrating aspects 1000 of identifying changesin an augmented reality scene, in accordance with an embodiment. Some orall of the process (or any other processes described, or variationsand/or combinations of those processes) may be performed under thecontrol of one or more computer systems configured with executableinstructions and/or other data, and may be implemented as executableinstructions executing collectively on one or more processors. Theexecutable instructions and/or other data may be stored on anon-transitory computer-readable storage medium (e.g., a computerprogram persistently stored on magnetic, optical, or flash media).

For example, some or all of process may be performed by any suitablesystem, such as a server in a data center, by various components of theenvironments described herein, including FIG. 13, such as the one ormore web servers 1306 or the one or more application servers 1308, bymultiple computing devices in a distributed system of a computingresource service provider, or by any electronic client device such asthe electronic client device 1302. The process includes a series ofoperations which, except where logically required (for example becausethe input to one step requires the output of a previous step, may beperformed in an alternative order, or performed in parallel. In someembodiments, certain steps may be omitted.

In 1002, a sequence of scene images is captured. In 1004, changesbetween frames are identified. This may comprise converting scene imagesinto a representation suitable for input to a neural network, or otherclassifier. For example, the sequence of image data might be convertedto a differential representation.

In 1006, a machine learning technique, such as an artificial neuralnetwork or classifier, is applied to answer one or more problems relatedto the identified changes in the scenes. The problems answerable byapplication of the machine learning are varied. For example, a networkmay be trained to identify frames of image data with significantchanges, such as a frame in which a person first enters into view, aframe in which a previously unrecognized object has appeared, and soforth. The output of the network can then be used to supply contextinformation for the service negotiation.

Note that, in the context of describing disclosed embodiments, unlessotherwise specified, use of expressions regarding executableinstructions (also referred to as code, applications, agents, etc.)performing operations that “instructions” do not ordinarily performunaided (e.g., transmission of data, calculations, etc.) denote that theinstructions are being executed by a machine, thereby causing themachine to perform the specified operations.

FIG. 11 is a flowchart illustrating an example process 1100 of servicenegotiation, in accordance with an embodiment. Some or all of theprocess (or any other processes described, or variations and/orcombinations of those processes) may be performed under the control ofone or more computer systems configured with executable instructionsand/or other data, and may be implemented as executable instructionsexecuting collectively on one or more processors. The executableinstructions and/or other data may be stored on a non-transitorycomputer-readable storage medium (e.g., a computer program persistentlystored on magnetic, optical, or flash media).

For example, some or all of process may be performed by any suitablesystem, such as an augmented reality device, or by a server in a datacenter, by various components of the environments described herein,including FIG. 13, such as the one or more web servers 1306 or the oneor more application servers 1308, by multiple computing devices in adistributed system of a computing resource service provider, or by anyelectronic client device such as the electronic client device 1302. Notethat the following process is generally described, in the followingdescription, in terms of actions taken by an augmented reality device.However, in various embodiments, some of the operations may be performedby an agent acting on behalf of the device. An agent process may, forexample, operate on a server in a data center. Likewise, an agentprocess may perform the actions attributed below to the second augmentedreality device.

The process includes a series of operations which, except wherelogically required (for example because the input to one step requiresthe output of a previous step, may be performed in an alternative order,or performed in parallel. In some embodiments, certain steps may beomitted.

At 1102, a first augmented reality device sends information for a seconddevice to perform a zero-knowledge proof of information associated withthe first augmented reality device. Examples of such informationinclude, but are not necessarily limited to, information indicative ofthe device's support for the negotiation protocol, and informationpertaining to the identity of an entity associated with the device.

The operation 1102 may occur in response to a request to initiate anegotiation process to the second device, where that request includesthe information necessary for the second device to perform thezero-knowledge proof.

At 1104, the first device receives information for performing azero-knowledge proof of a second entity associated with the seconddevice. In an embodiment, the first device receives a request to performthe proof from the second device, sent by the second device after itverified the proof requested by the first device.

More generally, the operations 1102 and 1104 may comprise mutuallyperformed proofs of information associated with the respective devices.In a negotiation process comprising a series of data exchanges withincreasing sensitivity, these proofs act, in embodiments, as the initialexchange.

At 1106, the first device responds to completion of the zero knowledgeproofs by providing the first entity's credentials to the second device.The second device may then use a third-party validation service toverify the first entity's credentials. Here, credentials may refer toinformation such as a user name and password, biometric information, orother information which uniquely identifies the first entity orotherwise permits the second device to determine a scope of accessrights appropriate to the circumstances.

At 1108, the first device validates the second party's credentials,using the third-party verification service. The credentials are sent, bythe second device, to the first device in response to the seconddevice's completion of the zero-knowledge proof of the first entity.

At 1110, the first device provides the second device with informationdescribing services that the first device is making available to thesecond device. These services are those that the second device isauthorized to use. Note that the prior operations involving thezero-knowledge proofs and credential verification may not, bythemselves, be determinative of the final set of access rights. Thenegotiation process may continue in a further series of exchanges, whichmay involve exchanges of data with escalating sensitivity, in order toestablish further contextual information that influences the final setof available services. For example, a user's privacy settings mightlimit the final set of available services, but this might depend oncontext and the identity of the other user. Thus, establishing the setof available services may depend on some combination of rights grantedby the respective credentials of the parties, the context of thesituation or transaction, and the preferences of the user.

Accordingly, in embodiments, the services authorized for a device arethose that conform to a mutually agreed level of sensitivity. Suchmutual authorization, in embodiments, comprises an indication of consentto provide the information by one party, and consent to receive theinformation by the other party. This mutual authorization may, inembodiments, be performed via a series of data interchanges that occurafter credentials have been verified at step 1108. For example, a firstdevice might indicate that it has and is willing to provide medicalinformation, to which the second device might respond that it is able toconsume such information. Further exchanges might establish agreed-uponparameters for the exchange. Similarly, a first device might indicatethat it is requesting sensitive information, while the second devicemight indicate that it is willing to provide such information, subjectto certain conditions. Further exchanges might indicate what thoseconditions are, and whether they are agreed upon. Upon completion ofthese exchanges, a set of authorized services might then be mutuallyagreed upon.

In these and other embodiments, authorization of services may refer tothose services which conform to a set of conditions.

In these and other embodiments, authorization of services may refer tothose services which conform to contextual requirements. Theserequirements may include one or more of, but are not necessarily limitedto, the time and place of the exchange, the identity of the entitiesinvolved, and the nature of a transaction being conducted orcontemplated.

FIG. 12 is a flowchart illustrating an example process of identifying asignal for service negotiation, in accordance with an embodiment. Someor all of the process (or any other processes described, or variationsand/or combinations of those processes) may be performed under thecontrol of one or more computer systems configured with executableinstructions and/or other data, and may be implemented as executableinstructions executing collectively on one or more processors. Theexecutable instructions and/or other data may be stored on anon-transitory computer-readable storage medium (e.g., a computerprogram persistently stored on magnetic, optical, or flash media).

For example, some or all of process may be performed by any suitablesystem, such as a server in a data center, by various components of theenvironments described herein, including FIG. 13, such as the one ormore web servers 1306 or the one or more application servers 1308, bymultiple computing devices in a distributed system of a computingresource service provider, or by any electronic client device such asthe electronic client device 1302. The process includes a series ofoperations which, except where logically required (for example becausethe input to one step requires the output of a previous step, may beperformed in an alternative order, or performed in parallel. In someembodiments, certain steps may be omitted.

The example process 1200 is employed, in embodiments, to determine whento initiate a service negotiation. As described herein, because theremay be numerous signals within the environment, it may be difficult toefficiently identify those signals which represent potential servicenegotiation partners. The example process 1200 uses an analysis of thewireless signals to identify an area transition effect that may occurwhen a user of a device crosses a threshold, as might happen when thatuser enters a building.

At 1202, an augmented reality device identifies one or more signals thatare exhibiting, or have exhibited, an area transition effect. An areatransition effect refers to various modulations or changes to a wirelesssignal that may be observed as a device receiving those signals crossesa threshold. As described herein, these changes may be observed, incases and embodiments, as a gradient in signal strength, such as anincrease or decrease in signal strength. In other cases an embodiments,the area transition effect is observed as a change in the strength ofthe signal across portions of a spectrum. For example, certain portionsof the spectrum occupied by a signal might become stronger, while otherportions become weaker. Moreover, some patterns may be indicative of thecorresponding device being carried into a room or building.

In embodiments, analysis of a plurality of signals in the ambientenvironment results in the selection of a subset of those signals, wherethe selected subset are those that have or are exhibiting the areatransition effect. At 1204, the augmented reality device selects, fromamong the signals exhibiting the area transition effect, a signal whichindicates that the device that is broadcasting the signal supports aservice negotiation protocol. For example, the augmented reality devicemight analyze the content of each of these signals to determine whichones contain an indication that they support the service negotiationprotocol. From these, a signal and its emitting device can be selectedfor engaging in a negotiation protocol. For convenience in description,this device can be referred to as the second device.

At 1206, the augmented reality device initiates a service negotiationprotocol with the second device. The negotiation may proceed, inembodiments, according to the procedure depicted in FIG. 11.

FIG. 13 illustrates aspects of an example environment 1300 forimplementing aspects in accordance with various embodiments. As will beappreciated, although a web-based environment is used for purposes ofexplanation, different environments may be used, as appropriate, toimplement various embodiments. The environment includes an electronicclient device 1302, which can include any appropriate device operable tosend and/or receive requests, messages, or information over anappropriate network 1304 and convey information back to a user of thedevice. Examples of such client devices include personal computers, cellphones, handheld messaging devices, laptop computers, tablet computers,set-top boxes, personal data assistants, embedded computer systems,electronic book readers, and the like.

The environment 1300 in one embodiment is a distributed and/or virtualcomputing environment utilizing several computer systems and componentsthat are interconnected via communication links, using one or morecomputer networks or direct connections. However, it will be appreciatedby those of ordinary skill in the art that such a system could operateequally well in a system having fewer or a greater number of componentsthan those illustrated in FIG. 9. Thus, the depiction in FIG. 13 shouldbe taken as being illustrative in nature and not limiting to the scopeof the disclosure.

The network 1304 can include any appropriate network, including anintranet, the Internet, a cellular network, a local area network, asatellite network or any other network, and/or combination thereof.Components used for such a system can depend at least in part upon thetype of network and/or environment selected. Many protocols andcomponents for communicating via such network 1304 are well known andwill not be discussed in detail. Communication over the network 1304 canbe enabled by wired or wireless connections and combinations thereof. Inan embodiment, the network 1304 includes the Internet and/or otherpublicly addressable communications network, as the environment 1300includes one or more web servers 1306 for receiving requests and servingcontent in response thereto, although for other networks an alternativedevice serving a similar purpose could be used as would be apparent toone of ordinary skill in the art.

The illustrative environment 1300 includes one or more applicationservers 1308 and data storage 1310. It should be understood that therecan be several application servers, layers or other elements, processesor components, which may be chained or otherwise configured, which caninteract to perform tasks such as obtaining data from an appropriatedata store. Servers, as used, may be implemented in various ways, suchas hardware devices or virtual computer systems. In some contexts,“servers” may refer to a programming module being executed on a computersystem. As used, unless otherwise stated or clear from context, the term“data store” or “data storage” refers to any device or combination ofdevices capable of storing, accessing, and retrieving data, which mayinclude any combination and number of data servers, databases, datastorage devices, and data storage media, in any standard, distributed,virtual, or clustered environment.

The one or more application servers 1308 can include any appropriatehardware, software and firmware for integrating with the data storage1310 as needed to execute aspects of one or more applications for theelectronic client device 1302, handling some or all of the data accessand business logic for an application. The one or more applicationservers 1308 may provide access control services in cooperation with thedata storage 1310 and is able to generate content including, text,graphics, audio, video, and/or other content usable to be provided tothe user, which may be served to the user by the one or more web servers1306 in the form of HyperText Markup Language (HTML), Extensible MarkupLanguage (XML), JavaScript, Cascading Style Sheets (CSS), JavaScriptObject Notation (JSON), and/or another appropriate client-sidestructured language. Content transferred to the electronic client device1302 may be processed by the electronic client device 1302 to providethe content in one or more forms including forms that are perceptible tothe user audibly, visually, and/or through other senses. The handling ofall requests and responses, as well as the delivery of content betweenthe electronic client device 1302 and the one or more applicationservers 1308, can be handled by the one or more web servers 1306 usingPHP: Hypertext Preprocessor (PHP), Python, Ruby, Perl, Java, HTML, XML,JSON, and/or another appropriate server-side structured language in thisexample. Further, operations described as being performed by a singledevice may, unless otherwise clear from context, be performedcollectively by multiple devices, which may form a distributed and/orvirtual system.

Each server typically will include an operating system that providesexecutable program instructions for the general administration andoperation of that server and typically will include a computer-readablestorage medium (e.g., a hard disk, random access memory, read onlymemory, etc.) storing instructions that, when executed (i.e., as aresult of being executed) by a processor of the server, allow the serverto perform its intended functions.

The data storage 1310 can include several separate data tables,databases, data documents, dynamic data storage schemes, and/or otherdata storage mechanisms and media for storing data relating to aparticular aspect of the present disclosure. For example, the datastorage 910 may include mechanisms for storing various types of data anduser information 1316, which can be used to serve content to theelectronic client device 1302. The data storage 1310 also is shown toinclude a mechanism for storing log data, such as application logs,system logs, access logs, and/or various other event logs, which can beused for reporting, analysis, or other purposes. It should be understoodthat there can be many other aspects that may need to be stored in thedata storage 1310, such as page image information and access rightsinformation, which can be stored in any of the above listed mechanismsas appropriate or in additional mechanisms in the data storage 1310. Thedata storage 1310 is operable, through logic associated therewith, toreceive instructions from the one or more application servers 1308 andobtain, update, or otherwise process data in response thereto. The oneor more application servers 1308 may provide static, dynamic, or acombination of static and dynamic data in response to the receivedinstructions. Dynamic data, such as data used in web logs (blogs),shopping applications, news services, and other applications may begenerated by server-side structured languages as described or may beprovided by a content management system (CMS) operating on, or under thecontrol of, the one or more application servers 1308.

In one embodiment, a user, through a device operated by the user, cansubmit a search request for a match to a particular search term. In thisembodiment, the data storage 1310 might access the user information toverify the identity of the user and obtain information about items ofthat type. The information then can be returned to the user, such as ina results listing on a web page that the user is able to view via abrowser on the electronic client device 1302. Information related to theparticular search term can be viewed in a dedicated page or window ofthe browser. It should be noted, however, that embodiments of thepresent disclosure are not necessarily limited to the context of webpages, but may be more generally applicable to processing requests ingeneral, where the requests are not necessarily requests for content.

The various embodiments further can be implemented in a wide variety ofoperating environments, which in some embodiments can include one ormore user computers, computing devices, or processing devices that canbe used to operate any of a number of applications. User or clientdevices can include any of a number of computers, such as desktop,laptop, or tablet computers running a standard operating system, as wellas cellular, wireless, and handheld devices running mobile software andcapable of supporting a number of networking and messaging protocols.Such a system also can include a number of workstations running any of avariety of commercially available operating systems and other knownapplications for purposes such as development and database management.These devices also can include other electronic devices, such as dummyterminals, thin-clients, gaming systems, and other devices capable ofcommunicating via the network 1304. These devices also can includevirtual devices such as virtual machines, hypervisors, and other virtualdevices capable of communicating via the network 904.

Various embodiments of the present disclosure utilize the network 1304that would be familiar to those skilled in the art for supportingcommunications using any of a variety of commercially availableprotocols, such as Transmission Control Protocol/Internet Protocol(TCP/IP), User Datagram Protocol (UDP), protocols operating in variouslayers of the Open System Interconnection (OSI) model, File TransferProtocol (FTP), Universal Plug and Play (UpnP), Network File System(NFS), and Common Internet File System (CIFS). The network 904 can be,for example, a local area network, a wide-area network, a virtualprivate network, the Internet, an intranet, an extranet, a publicswitched telephone network, an infrared network, a wireless network, asatellite network, and any combination thereof. In some embodiments,connection-oriented protocols may be used to communicate between networkendpoints. Connection-oriented protocols (sometimes calledconnection-based protocols) are capable of transmitting data in anordered stream. Connection-oriented protocols can be reliable orunreliable. For example, the TCP protocol is a reliableconnection-oriented protocol. Asynchronous Transfer Mode (ATM) and FrameRelay are unreliable connection-oriented protocols. Connection-orientedprotocols are in contrast to packet-oriented protocols such as UDP thattransmit packets without a guaranteed ordering.

In embodiments utilizing the one or more web servers 1306, the one ormore web servers 1306 can run any of a variety of server or mid-tierapplications, including Hypertext Transfer Protocol (HTTP) servers, FTPservers, Common Gateway Interface (CGI) servers, data servers, Javaservers, Apache servers, and business application servers. The server(s)also may be capable of executing programs or scripts in response torequests from user devices, such as by executing one or more webapplications that may be implemented as one or more scripts or programswritten in any programming language, such as Java® C, C# or C++, or anyscripting language, such as Ruby, PHP, Perl, Python, or TCL, as well ascombinations thereof. The server(s) may also include database servers,including those commercially available from Oracle®, Microsoft®,Sybase®, and IBM® as well as open-source servers such as MySQL,Postgres, SQLite, MongoDB, and any other server capable of storing,retrieving, and accessing structured or unstructured data. Databaseservers may include table-based servers, document-based servers,unstructured servers, relational servers, non-relational servers, orcombinations of these and/or other database servers.

The environment 1300 can include a variety of data stores and othermemory and storage media as discussed above. These can reside in avariety of locations, such as on a storage medium local to (and/orresident in) one or more of the computers or remote from any or all ofthe computers across the network 1304. In a particular set ofembodiments, the information may reside in a storage-area network (SAN)familiar to those skilled in the art. Similarly, any necessary files forperforming the functions attributed to the computers, servers or othernetwork devices may be stored locally and/or remotely, as appropriate.Where a system includes computerized devices, each such device caninclude hardware elements that may be electrically coupled via a bus,the elements including, for example, a central processing unit (CPU orprocessor), an input device (e.g., a mouse, keyboard, controller, touchscreen, or keypad), and an output device (e.g., a display device,printer, or speaker). Such a system may also include one or more storagedevices, such as disk drives, optical storage devices, and solid-statestorage devices such as random access memory (RAM) or read-only memory(ROM), as well as removable media devices, memory cards, flash cards,etc.

Such devices also can include a computer-readable storage media reader,a communications device (e.g., a modem, a network card (wireless orwired), an infrared communication device, etc.), and working memory asdescribed above. The computer-readable storage media reader can beconnected with, or configured to receive, a computer-readable storagemedium, representing remote, local, fixed, and/or removable storagedevices as well as storage media for temporarily and/or more permanentlycontaining, storing, transmitting, and retrieving computer-readableinformation. The system and various devices also typically will includea number of software applications, modules, services, or other elementslocated within a working memory device, including an operating systemand application programs, such as a client application or web browser.In addition, customized hardware might also be used and/or particularelements might be implemented in hardware, software (including portablesoftware, such as applets), or both. Further, connection to othercomputing devices such as network input/output devices may be employed.

Storage media and computer readable media for containing code, orportions of code, can include any appropriate media known or used in theart, including storage media and communication media, such as, volatileand non-volatile, removable and non-removable media implemented in anymethod or technology for storage and/or transmission of information suchas computer readable instructions, data structures, program modules, orother data, including RAM, ROM, Electrically Erasable ProgrammableRead-Only Memory (EEPROM), flash memory or other memory technology,Compact Disc Read-Only Memory (CD-ROM), digital versatile disk (DVD), orother optical storage, magnetic cassettes, magnetic tape, magnetic diskstorage, or other magnetic storage devices, or any other medium whichcan be used to store the desired information and which can be accessedby the system device. Based on the disclosure and teachings provided, aperson of ordinary skill in the art will appreciate other ways and/ormethods to implement the various embodiments.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. However, it will beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the invention asset forth in the claims. Other variations are within the spirit of thepresent disclosure. Thus, while the disclosed techniques are susceptibleto various modifications and alternative constructions, certainillustrated embodiments thereof are shown in the drawings and have beendescribed above in detail. It should be understood, however, that thereis no intention to limit the invention to the specific form or formsdisclosed, but on the contrary, the intention is to cover allmodifications, alternative constructions, and equivalents falling withinthe spirit and scope of the invention, as defined in the appendedclaims.

The use of the terms “a,” “an,” “the,” and similar referents in thecontext of describing the disclosed embodiments (especially in thecontext of the following claims) are to be construed to cover both thesingular and the plural, unless otherwise indicated or clearlycontradicted by context. The terms “comprising,” “having,” “including,”and “containing” are to be construed as open-ended terms (i.e., meaning“including, but not limited to,”) unless otherwise noted. The term“connected,” where unmodified and referring to physical connections, isto be construed as partly or wholly contained within, attached to, orjoined together, even if there is something intervening. Recitation ofranges of values are merely intended to serve as a shorthand method ofreferring individually to each separate value falling within the range,unless otherwise indicated and each separate value is incorporated intothe specification as if it were individually recited. The use of theterm “set” (e.g., “a set of items”) or “subset” unless otherwise notedor contradicted by context, is to be construed as a nonempty collectioncomprising one or more members. Further, unless otherwise noted orcontradicted by context, the term “subset” of a corresponding set doesnot necessarily denote a proper subset of the corresponding set, but thesubset and the corresponding set may be equal.

Conjunctive language, such as phrases of the form “at least one of A, B,and C,” or “at least one of A, B and C,” is understood with the contextas used in general to present that an item, term, etc., may be either Aor B or C, or any nonempty subset of the set of A and B and C, unlessspecifically stated otherwise or otherwise clearly contradicted bycontext. For instance, in the illustrative example of a set having threemembers, the conjunctive phrases “at least one of A, B, and C” and “atleast one of A, B and C” refer to any of the following sets: {A}, {B},{C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Thus, such conjunctive languageis not generally intended to imply that certain embodiments require atleast one of A, at least one of B and at least one of C each to bepresent. In addition, unless otherwise noted or contradicted by context,the term “plurality” indicates a state of being plural (e.g., “aplurality of items” indicates multiple items). The number of items in aplurality is at least two, but can be more when so indicated eitherexplicitly or by context.

Operations of processes described can be performed in any suitable orderunless otherwise indicated or otherwise clearly contradicted by context.Processes described (or variations and/or combinations thereof) may beperformed under the control of one or more computer systems configuredwith executable instructions and may be implemented as code (e.g.,executable instructions, one or more computer programs or one or moreapplications) executing collectively on one or more processors, byhardware or combinations thereof. The code may be stored on acomputer-readable storage medium, for example, in the form of a computerprogram comprising instructions executable by one or more processors.The computer-readable storage medium may be non-transitory. In someembodiments, the code is stored on a set of one or more non-transitorycomputer-readable storage media having stored thereon executableinstructions that, when executed (i.e., as a result of being executed)by one or more processors of a computer system, cause the computersystem to perform operations described herein. The set of non-transitorycomputer-readable storage media may comprise multiple non-transitorycomputer-readable storage media and one or more of individualnon-transitory storage media of the multiple non-transitorycomputer-readable storage media may lack all of the code while themultiple non-transitory computer-readable storage media collectivelystore all of the code. Further, in some embodiments, the executableinstructions are executed such that different instructions are executedby different processors. As an illustrative example, a non-transitorycomputer-readable storage medium may store instructions. A main CPU mayexecute some of the instructions and a graphics processor unit mayexecute other of the instructions. Generally, different components of acomputer system may have separate processors and different processorsmay execute different subsets of the instructions.

Accordingly, in some embodiments, computer systems are configured toimplement one or more services that singly or collectively performoperations of processes described herein. Such computer systems may, forinstance, be configured with applicable hardware and/or software thatenable the performance of the operations. Further, computer systems thatimplement various embodiments of the present disclosure may, in someembodiments, be single devices and, in other embodiments, be distributedcomputer systems comprising multiple devices that operate differentlysuch that the distributed computer system performs the operationsdescribed and such that a single device may not perform all operations.

The use of any examples, or exemplary language (e.g., “such as”)provided, is intended merely to better illuminate embodiments of theinvention and does not pose a limitation on the scope of the inventionunless otherwise claimed. No language in the specification should beconstrued as indicating any non-claimed element as essential to thepractice of the invention.

Embodiments of this disclosure are described, including the best modeknown to the inventors for carrying out the invention. Variations ofthose embodiments may become apparent to those of ordinary skill in theart upon reading the foregoing description. The inventors expect skilledartisans to employ such variations as appropriate and the inventorsintend for embodiments of the present disclosure to be practicedotherwise than as specifically described. Accordingly, the scope of thepresent disclosure includes all modifications and equivalents of thesubject matter recited in the claims appended hereto as permitted byapplicable law. Moreover, although above-described elements may bedescribed in the context of certain embodiments of the specification,unless stated otherwise or otherwise clear from context, these elementsare not mutually exclusive to only those embodiments in which they aredescribed; any combination of the above-described elements in allpossible variations thereof is encompassed by the scope of the presentdisclosure unless otherwise indicated or otherwise clearly contradictedby context.

All references, including publications, patent applications, andpatents, cited are hereby incorporated by reference to the same extentas if each reference were individually and specifically indicated to beincorporated by reference and were set forth in its entirety.

Embodiments of the present disclosure may be further understood in viewof the following clauses.

In various embodiments, a system, method, or non-transitorycomputer-readable medium performs, includes, or causes to be performedoperations to negotiate available computing services, in accordance withthe techniques disclosed herein.

In various embodiments, a system, method, or non-transitorycomputer-readable medium performs, includes, or causes to be performedoperations to display face-associated data in an augmented realitydisplay, in accordance with the techniques disclosed herein.

In various embodiments, a system, method, or non-transitorycomputer-readable medium performs, includes, or causes to be performedoperations to identify differences in a sequence of image data andanswer questions based on the identified differences.

What is claimed is:
 1. A device, comprising: a receiver; at least oneprocessor; and a memory having stored thereon executable instructionsthat, in response to execution by the at least one processor, cause thedevice to: identify, from among a plurality of signals received by thereceiver, one or more signals associated with an area transition effect;select a first signal from among the one or more signals, based at leastin part on determining that the first signal comprises informationindicative of a second device that supports negotiation of services; anddetermine a set of services to make available to the second device,based at least in part on the negotiation of services, wherein at leastone service, of the set of services, is a service for customizing anaugmented reality experience of a user of the device, the augmentedreality experience customized based on information provided by thesecond device using the at least one service.
 2. The device of claim 1,wherein the area transition effect comprises a rate of signal strengthincrease that is above a threshold level.
 3. The device of claim 1,wherein the area transition effect is indicative of the device crossinga threshold.
 4. The device of claim 1, wherein the memory has storedthereon executable instructions that, in response to execution by the atleast one processor, cause the device to: determine that the firstsignal comprises an identifier; retrieve stored information thatassociates the identifier with a geographic location; and determine thatthe geographic location associated with the identifier corresponds to acurrent location of the device.
 5. The device of claim 1, wherein thememory has stored thereon executable instructions that, in response toexecution by the at least one processor, cause the device to: determinethat the first signal comprises information indicative of a request toinitiate a negotiation of services.
 6. The device of claim 1, wherein alevel of authorized interaction with the device is determined based atleast in part on a negotiation protocol.
 7. A method, comprising:identifying, from among a plurality of received signals, one or moresignals associated with an area transition effect; selecting a firstsignal from among the one or more signals, based at least in part ondetermining that the first signal comprises information indicative of asecond device supporting negotiation of services; and determining one ormore services to make available to the second device, based at least inpart on a negotiation of services with the second device, wherein atleast one service, of the one or more services to make available to thesecond device, is a service for customizing an augmented realityexperience of a user of the second device, based on information providedby the second device.
 8. The method of claim 7, wherein the areatransition effect comprises a rate of signal strength increase that isabove a threshold level.
 9. The method of claim 7, wherein the areatransition effect is indicative of the second device being carriedthrough a threshold.
 10. The method of claim 7, further comprising:filtering a plurality of received signals to identifying a subset ofsignals to analyze for area transition effects.
 11. The method of claim7, further comprising: obtaining, from a database, a geographic locationassociated with an identifier obtained from the first signal;correlating the geographic location associated with the identifier witha location of the second device; and determining, based at least in parton the identifier and the correlating, to initiate the negotiation ofservices.
 12. The method of claim 7, further comprising: determiningthat one or more of the signals associated with the area transitioneffect comprise information indicative of a request to initiate anegotiation of services.
 13. The method of claim 7, further comprising:determining a level of authorized interaction with the second device.14. A non-transitory computer-readable storage medium having storedthereon executable instructions that, in response to being executed byone or more processors of a computing device, cause the computing deviceto at least: identify, from among a plurality of received signals, oneor more signals associated with an area transition effect; select afirst signal from among the one or more signals, based at least in parton determining that the first signal comprises information indicative ofa second device supporting negotiation of services; and determine one ormore services to make available to the second device, based at least inpart on a negotiation of services with the second device, wherein atleast one service, of the one or more services to make available to thesecond device, is a service for customizing an augmented realityexperience of a user of the second device, based on information providedby the second device.
 15. The non-transitory computer-readable storagemedium of claim 14, wherein the area transition effect comprises a rateof signal strength increase that is above a threshold level.
 16. Thenon-transitory computer-readable storage medium of claim 14, wherein thearea transition effect is indicative of the device being carried througha threshold.
 17. The non-transitory computer-readable storage medium ofclaim 14, having stored thereon further executable instructions that, inresponse to being executed by one or more processors of an augmentedreality device, cause the computing device to at least: identify thetransition effect based at least in part on filtering of a plurality ofsignals.
 18. The non-transitory computer-readable storage medium ofclaim 14, having stored thereon further executable instructions that, inresponse to being executed by one or more processors of the computingdevice, cause the computing device to at least: obtain, from a database,a geographic location associated with an identifier obtained from thefirst signal; correlate the geographic location associated with theidentifier with a location of the computing device; and determine, basedat least in part on the identifier and the correlation, to initiate thenegotiation of services.
 19. The non-transitory computer-readablestorage medium of claim 14, wherein the first signal is selected basedat least in part on the first signal comprising a request to initiate anegotiation of services.
 20. The non-transitory computer-readablestorage medium of claim 14, having stored thereon further executableinstructions that, in response to being executed by one or moreprocessors of an augmented reality device, cause the computing device toat least: determine, based at least in part on location, informationthat may be shared with the one or more services.